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Abstract 

This descriptive research study examined 2013 NAEP 12th-grade mathematics scores by 
students’ use of graphing calculators in math classes and the kind of calculator students used 
during NAEP assessment. NAEP Data Explorer analysis included two questions from Student 
Factors: How often do you use these different kinds of calculators in math class? Graphing 
[M817601], and Kind of calculator used for test? [M830201]. The literature review included 
international experimental and quasi-experimental research on student use of calculators during 
learning and assessment. The overall (100%) M=152, SD=34 (scale-range 0-300). NAEP scores 
by use of graphing calculator in math class were: never use (18%) M=137, SD=29; sometimes 
not often use (21%) M=145, SD=34; usually use (61%) M=165, SD=32. NAEP scores by 
calculator use for the test: None: (35%) M=153, SD=33; NAEP provided: (39%) M=147, 
SD=30; Scientific non-graphing: (8%) M=146, SD=33, Graphing: (17%) M=171, SD=33. Effect 
sizes (Cohen, 1988) ranged from d=0.26 to <7=0.90. 
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Impact of Student Calculator Use on the 2013 NAEP Twelfth-Grade Mathematics 

Assessment 

The use of calculators in teaching mathematics in elementary and secondary schools in 
the United States has been a variable for instruction and assessment across the years. Robelen 
(2013), citing Horizon Research, Inc., stated that 64% of high school students use graphing 
calculators at least once a week (p. 12). Robelen suggested that the use of graphing calculators in 
teaching and testing in the United States would change with the coming Core-aligned exams. 

The use of calculators will continue to be permitted in the international assessments 
Program for International Student Assessment (PISA) (Organization for Economic Co-Operation 
and Development (OECD), March, 2013), and Trends in International Mathematics and Science 
Study (TIMSS). TIMMS calculator policy for the eighth-grade level was described by Mullis and 
Martin (2013): 

Calculators and policies for their use differ across the TIMSS countries. Also, the 
availability of calculators varies widely. It would not be equitable to require 
calculator use when students in some countries may never have used them. 
Similarly, however, it is not equitable to deprive students of the use of a familiar 
tool. In order to give students the best opportunity to operate in settings that 
mirror their classroom experience, TIMSS has permitted calculator use at the 
eighth grade since 2003. Thus, if eighth grade students are accustomed to having 
calculators for their classroom activities, then the country should encourage 
students to use them during the assessment. On the other hand, if students are not 
accustomed to having calculators or are not permitted to use them in 
their daily mathematics lessons, then the country need not permit their use. In 
developing the new assessment materials, every effort will be made to ensure 
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that the test questions do not advantage or disadvantage students either way, 
with or without calculators, (p. 24) 

Calculator use permitted on NAEP fourth-, eithth-, and twelfth-grade mathematics 
assessments varies by grade level. National Center for Educational Statistics (NCES) 

(November, 2013) stated: 

NAEP Calculator Policy 

The mathematics assessment contains some sections for which calculators are not 
allowed, and other sections that contain some questions that would be difficult to 
solve without a calculator. At each grade level, approximately two-thirds of the 
assessment measures students’ mathematical knowledge and skills without access 
to a calculator; the other third allow a calculator’s use. The type of calculator 
students may use varies by grade level, as follows: 

- At grade 4, a four-function calculator is supplied to students, with 
training at the time of administration. 

- At grades 8 and 12, students are allowed to bring whatever 
calculator, graphing or otherwise, they are accustomed to using in 
the classroom with some restrictions for test security purposes. For 
students who do not bring a calculator to use on the assessment, 
NAEP will provide a scientific calculator. 

No questions in the test are designed to provide an advantage to students 
with a graphing calculator. Questions are categorized according to the 
degree to which a calculator is useful in responding to the item: 

■ A calculator inactive question is one whose solution neither 


requires nor suggests the use of a calculator. 
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- A calculator is not necessary for solving a calculator neutral 
question; however, given the option, some students might choose 
to use one. 

- A calculator is necessary or very helpful in solving a calculator 
active question; a student would find it very difficult to solve the 
problem without the aid of a calculator, (para 10-11) 

The Purpose of the Study 

The purpose of this quantitative descriptive research study was to examine whether or 
not there were differences in 2013 NAEP twelfth-grade mathematics average scale scores by: (1) 
students’ use of a graphing calculator in math classes, and (2) kind of calculator the students 
used for the NAEP assessment. 

This secondary analysis of the NAEP data used the national public data composite 
mathematics scale scores from the 2013 NAEP twelfth-grade mathematics assessment. NAEP 
Data Explorer was used for analyses. Two questions were selected using: Student Factors 
instructional Content and Practice> Modes of instruction/classroom activities from the Select 
Variables option in the NAEP Data Explorer (NCES, 2014a): 

(1) How often do you use these different kinds of calculators in math 
class? Graphing. Options: never use; sometimes not often; usually use 
(answered by students). [M817601] 

(2) Kind of calculator used for test? Options: no calculator use; scientific non- 
graphing calculator; NAEP provided calculator; graphing calculator (answered by 


students). [M830201] 
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Review of Literature 

The review of literature begins with studies that examined the impact of the use of 
graphing calculators during learning followed by research that examined the impact of calculator 
use during assessments. 

Graphing/Graphic Calculator Use During Learning 

Lukens and Feinstein (2000) studied the impact of the use of graphing calculators in 
teaching on Advanced Placement Biology exam scores: 

Students enrolled in the integrated Advanced Placement Biology/Advanced 
Placement Calculus block course where graphing calculators were a meaningful 
part of the curriculum performed significantly better on the Advanced Placement 
Biology exam when compared to students enrolled in the traditional Advanced 
Placement Biology where graphing calculators were not heavily integrated into 
the course. There appears to be a positive correlation between integrating 
graphing calculators in the higher level biology classes and the understanding and 
achievement of the students enrolled in the integrated course, (p. 1) 

Hasan, Azizan, & Kassim (December, 2005) described the introduction of graphic 
calculators into the Malaysian New Curriculum for secondary schools Form 4(16 years old). 

The students came from low-income families and had little access to technology. The researchers 
used an experimental control-group design with use of graphic calculators (TI-83 Plus) as the 
manipulated independent variable. They found that the mean post-test of the group using graphic 
calculators (M=9.83, SD 2.678) was higher than the post-test mean of the control group 
(M=3.86, SD=2.248) (T 3 p-value =0.000). 

In a similar study Nor’ain Mohd. Tajudin & Noraini Idris (2013) used a quasi- 
experimental non-nonequivalent control group design to compare the effect between the TI- 
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Nspire CX graphing calculator (CG) and conventional instruction (Cl) on learning Straight Lines 
and Statistics. Two intact classrooms of students were selected from four schools in Perak and 
Selangor. One class in each school was assigned the experimental treatment; the other served as 
the control. They found that students in the experimental group from all schools had significantly 
higher scores on the outcome measures than the students from the control group. They concluded 
that TI-Nspire has the potential to promote higher-level mathematical thinking. 
Graphing/Graphic Calculator Use During Assessment 

Haimer (1999) examined calculator use and equity on the 1998 Tertiartiary Entrance 
Exam (TEE) in Australia. This exam required the use of graphic calculators by all students. 
Haimer examined gender equity and hypothesized that females would have more difficulty than 
males using graphing calculators. The hypothesis was not supported. A later secondary analysis 
of the same data by Haimer and Webster (2001) compared the mathematics scores of rural and 
urban Western Australian students. Urban students scored higher than rural students in the three 
years studied; however, the gap was inconsistent across items. Haimer and Webster (2001) 
concluded that rural students were not further disadvantaged by the requirement of graphic 
calculator use for the test. 

Schwarz, Rich, Arenson, Podrabsky & Cook (2002) explored the question of differences 
by calculator use in evaluation with data from the Tennessee Gateway assessment given as an 
end-of course test in Algebra 1. Data were from approximately 7,000 students who answered 
questions about their use of calculators. Response options were “use” or “no use” and type of 
calculator. Schwartz et al. (2002) found: 

Calculator type, usage, and familiarity were associated with differences in the 
univariate comparison of test scores. For instance, students who responded that a 
graphing calculator was used performed higher than the other groups. The use of a 
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graphing calculator could indicate that higher-level mathematics courses had been 
taken. However, this may not be the case in this instance since all students take 
the Gateway examination at the conclusion of their first Algebra I class, (p. 16) 
DeLoach (2013) used a quantitative, quasi-experimental study to compare the 
performance of Algebra II students. Students in the experimental group used a graphing 
calculator to complete a standardized mathematics test. Students in the control group did not use 
a graphing calculator to complete the same test. The results of the independent /-test found that 
the mean of the experimental group was higher than the mean of the control group ( t (51) = 2.69, 

p=.01). 

Ellington (2006) conducted a meta-analysis of research that had investigated the 
achievement of students using Non-CAS Graphing Calculators. She summarized in the Abstract: 
Forty-two studies comparing students with access to graphing calculators during 
instruction to students who did not have access to graphing calculators during 
instruction are the subject of this meta-analysis. The results on the achievement 
and attitude levels of students are presented. The studies evaluated cover middle 
and high school mathematics courses, as well as college courses through first 
semester calculus. When calculators were part of instruction but not testing, 
students' benefited from using calculators while developing the skills necessary to 
understand mathematics concepts. When calculators were included in testing and 
instruction, the procedural conceptual and overall achievement skills of students 


improved, (p. 2) 
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Large-Scale Study with Scientific Non-Graphing Calculators 

Close, Oldham, Shiel, Dooley, & O’Leary (2012) described an extended study with 1,469 
Grade nine students; however, the type of calculator was “scientific.” The authors described the 
study: 

Three calculator tests were administered to a national sample of 1,469 Irish 
students in Grade 9-the last cohort to study mathematics without calculators 
(Phase 1). Three years later, the same tests were administered to a similar sample 
with calculators (Phase 2). Scores on a test of calculator-inappropriate items 
showed no significant change over the 3 years. For a test of calculator optional 
items, students were divided randomly into 2 groups, 1 with calculator access and 
the other without. In both phases, the students with calculators achieved 
significantly better than the students without calculators. Achievement on a test of 
calculator appropriate items showed significant improvement over the 3 years. 
Students’ attitudes toward calculators also improved over the time. (p. 377) 
However, the types of calculators used by the Grade 9 students in the study were not well- 
defined. The authors stated: 

Students ’ use of calculators, [italicized heading in the original] In Phase 1, 55.8% 
of students reported that they had access to a calculator at school, whereas 
practically all students in Phase 2 had access to one. By Phase 2, scientific 
calculators were most frequently owned, and fewer than 1% of students used or 
had access to a graphing calculator, (p. 385) 

Summary 

The literature reviewed included correlational and quantitative experimental and quasi- 
experimental designs that investigated the effects of graphing (graphic) calculators both in 
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teaching and assessment. The large-scale Irish study that was conducted across time included 
mostly “scientific” calculators and very few (about 1%) graphing calculators. In the literature 
reviewed higher assessment scores were obtained by students who used graphing (graphic) 
calculators for learning and/or during assessments. 

Method 

This secondary analysis of NAEP data used a quantitative descriptive research design that 
examined differences in the average scale scores (composite) by the two research questions. 

Participants and Sampling 

Because state NAEP assessments do not include 12 th - grade students, a grade twelve 
sample of schools was selected (NCES, 2014b). The sample was designed to provide national 
estimates of 12 th -grade achievement. The 2009 and 2013 samples were from selected students 
within selected schools from eleven volunteer states: Arkansas, Connecticut, Florida, Idaho, 
Illinois, Iowa, Massachusetts, New Hampshire, New Jersey, South Dakota, and West Virginia 
(NCES, 2014b). The target population (rounded to nearest thousand) for the national public data 
was 2,986,000. The sample for the national public data (rounded to nearest hundred) was 46,000 
(U.S. Department of Education, 2013, p.l). 

Data Analysis 

The NAEP Data Explorer (NCES 2014a) was used for the data analysis. The 2013 
NAEP twelfth-grade mathematics assessment composite average scale scores and standard 
deviations were selected for the analyses. The three coded questions selected through Data 
Explorer (NCES, 2014a) were: 

(1) How often do you use these different types of calculators in your math 
class? Graphing. Options: never use; sometimes not often; usually use 


(answered by students). [M817601] 
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(3) Kind of calculator used for test? Options: no calculator use; scientific non- 
graphing calculator; NAEP provided calculator; graphing calculator (answered by 
students). [M830201] 

Descriptive tables and tests of statistically significant of differences (alpha set a priori at 
0.001 because of the large n of the sample) were calculated and presented by Data Explorer 
(NCES, 2014a). In several instances, the tables were re-formatted without editing the data in the 
tables. Cohen’s d effect sizes (Cohen, 1988) were hand calculated. 

Results 

Missing Frequency (N) in Tables 

The NAEP Data Explorer tool (NCES, 2014a) does not include the number of students 
either in the overall data (100%) or within sub- strata of the data. Consequently, tables in this 
Results section include the percentages of the sample that are in the sub-strata reported without 
the expected frequency (N). 

The average scale score mean for 100% of the twelfth-grade students on the 2013 
NAEP Mathematics assessment was 152 (scale-range 0-300) with a standard deviation of 34. 
Differences in scores by questions are presented in tables throughout the results section. 

[place Table 1 about here] 

The average scale score of the twelfth-grade students who reported that they never 
used graphing calculators (18%) was 137 (29). The mean of the average scale score of the 
students who “sometimes not often” used graphing calculators in math class (21%) was 145 (34). 
The average scale score of the students who “usually use” graphing calculators in math class 
(61%) was 165 (32) (Table 1). 


[place Table 2 about here] 
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Table 2, created by NAEP Data Explorer, presents mean differences and the results of 
multiple independent /-tests. Note that NAEP Data Explorer analyses have alpha set at 0.05 
rather than the 0.001 set a priori by the researchers. The average scale scores of students (21%) 
who “sometimes not often” used graphing calculators in math class (M=145; SD=34) was 
significantly (p <0.001) higher than the average scale scores of students (18%) who “never 
used” graphing calculators in math class (M=137; SD=29). The average scale scores students 
(61%) who “usually use” graphing calculators in math class (M=165, SD=32) was 20 points 
higher than students who “sometimes not often” used graphing calculators in math class and 28 
points higher than students who “never used” graphing calculators in mathematics classes. 

[place Table 3 about here] 

The effect size of the difference between the mean average scale score of students who 
reportedly “usually use” graphing calculators in math class and the mean for students who “never 
use” graphing calculators in math class was d= 0.90 (Table 3). The effect size of the difference 
between mean average scale scores of students who “usually use” and students who “sometimes 
not often” use graphing calculators in math class was c/=0.60. The effect size of the difference 
between mean average scale scores of students who “sometimes not often” and “never use” 
graphing calculators in math class was cl= 0.26. The effect sizes were small (0.26), medium 
(0.60), and large (0.90). 

The large effect size ( d = 0.90) was between the mean scale scores of students (18%) who 
“never use” used graphing calculators in math class and the mean average scale scores of 
students (61%) who “usually use” graphing calculators in math class. This finding is consistent 
with the findings of studies reviewed for this study (e.g., Hasam, Azizan, &Kassim, December, 
2005; Lukens & Feinstein, 2000; Nor’am Mohd, Tajudi, & Noraini Idris, 2013). 
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There is a dissonance between the data in Table 1 and Table 4 (below). From the data in 
Table 1, 61% of the students reported that they “usually use” graphing calculators in 
mathematics class. However, only 17% of the students indicated that they used graphing 
calculators (Table 4) to take NAEP 2013 twelfth-grade mathematics assessment. 

[place Table 4 about here] 

The average scale score for the 35% of the twelfth-grade students who used no calculator 
for the test was 153 (33) (Table 4). The average scale score for the 39% of the students who used 
the NAEP-Provided calculator was 147 (32). The average scale score for the 8% of the students 
who used a Scientific Not Graphing calculator was 146 (31). The average scale score for the 
17% of the students who used a Graphing calculator was 171 (33) (Table 4). 

[place Table 5 about here] 

Table 5 presents results of the NAEP Data Explorer test for statistical significance 
between the average score means by calculator-use category. There was no statistically 
significant difference between the average scale scores of the students who used the NAEP- 
provided calculator (M=146) and the students who used the Scientific Non-Graphing calculator 
(M=147), (p = 0.4592). The NAEP-provided calculator was described as a scientific-non- 
graphing calculator, thus, this result is not surprising. Means for each of these categories were 
significantly (p <0.001) lower than the mean for students reporting that they used no calculator 
(M=153). 

The average scale score of the students who used a graphing calculator for the NAEP 
assessment was significantly different from the average scale scores of students who selected 
other categories (Table 5). The scores of students using graphing calculators was 18 point higher 
than average scale scores of students using no calculator, 24 points higher than the average scale 
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scores of students using NAEP-provided calculators, and 25 points higher than the average scale 
scores of students using scientific non-graphing calculators. 

[place Table 6 about here] 

The effect size for the difference in the average scale score of students using graphing 
calculators and the average scale score of students using no calculator was <7=0.55 (Table 6). 

The effect size of the difference between the average scale score of students using the graphing 
calculator and the average scale score of students using the NAEP-provided calculator was 
<7=0.74. The effect size for the difference in the average scale score of students who used 
graphing calculators and the average scale score of students who used scientific, non-graphing 
calculators was d= 0.78. The effect sizes were medium (<7=0.55) to large ( c/=0.74 and <7=0.78) 
(Cohen, 1988). 

[place Table 7 about here] 

Table 7 presents the means and standard deviations of the NAEP 2013 twelfth-grade 
mathematics scores by Kind of Calculator Used for the NAEP Test by Graphic Calculator Used 
in Math Class. The highest average scale score on Table 7 (M=177, SD=29) is for the students 
who “usually used” a graphing calculator for math class and who also used a graphing calculator 
while taking the NAEP test. This score was 1 1 points higher (effect size <7=0.37) than the next 
highest average scale score (M=166, SD=31) obtained by students who “usually use” a graphing 
calculator in math class and used no calculator on the NAEP test. 

The group of students who “usually used” a graphing calculator in math class and used a 
graphing calculator while taking the NAEP twelfth-grade 2013 mathematics exam exceeded the 
overall average score (M=152, SD=34) by 25 points with a large effect size of d= 0.79. 
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Discussion 

This paper was inspired by a conversation with colleagues about equity in student 
technology use in public schools. A question was raised, “Is there research evidence that the use 
of technology causes increased learning?” With “technology” more narrowly defined as 
calculator use in mathematics learning and assessment, a review of experimental and quasi- 
experimental research studies found that the use of graphing calculators in mathematics teaching 
and assessment in secondary schools caused higher student achievement. 

The NAEP 2013 twelfth-grade mathematics public school data were selected for the 
study for three reasons: (1) the data were from a large representative sample of twelfth-grade 
students from the United States, (2) the database included information about secondary students’ 
calculator use in the classroom and during the NAEP test, and (3) the researchers’ familiarity 
with NAEP data, the NAEP Data Explorer, and the use of graphing calculators in math class and 
assessments. 

Summary of Findings 

1. The average scale score of students (61% ) who “usually used” graphing 
calculators in math class was 20 point higher than students who “sometimes not 
often used” graphing calculators in math class (effect size <7=0.60) and 28 points 
higher than students who “never used” graphing calculators in math class (effect 
size <7=0.90). 

2. A noted dissonance in the data, overall 61% of the students reported that they 
“usually used” graphing calculators for math class; 17% of the students reported 
that they used graphing calculators for the test. 
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3. The average scale score of students (17%) who used graphing calculators for the 
test was: 

• 18 points higher the than average scale score of students (35%) who did 
not use a calculator for the test (effect size J=0.55 ) 

• 24 points higher than the average scale score of students (39%) who used 
the NAEP-provided calculator (effect size <7=0.74) 

• 25 points higher than the average scale score of students (8%) who used a 
scientific non-graphing calculator (effect size <7=0.78) 

4. The average scale score of students who “usually used” a graphing calculator in 
math class and used a graphing calculator while taking the NAEP twelfth-grade 
2013 mathematics assessment (M=177, SD=29) (Table 5) exceeded the overall 
average score (M=152, SD=34) by 25 points with a large effect size of <7=0.79. 

Conclusions 

Results of this quantitative descriptive research study indicate that the average scale 
scores of 12 th -grade students who “usually use” graphing calculators in math class (61%) are 
higher than students who do not use graphing calculators in math class. Additionally, average 
scale scores of students who used a graphing calculator while taking the 2013 NAEP 12 th - grade 
Mathematics assessment had higher scores than students who used no calculator or another type 
of non-graphing calculator. Further, the smaller group of students who both “usually use” a 
graphing calculator in math class and used a graphing calculator for the test had an average scale 
score that was higher than that of other students who “usually use” a graphing calculator in math 
class but used either no calculator of another kind of non-graphing calculator for the test. The 
effect size of the difference between the scores of the students who both usually used a graphing 
calculator in mathematics class and also used a graphing calculator for the test (Table 
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These findings are consistent with the results of the experimental and quasi-experimental 
studies in the review of literature (DeLoach , 2013; Hasan, Azizan, & Kassim, December, 2005; 
Nor’ain Mohd. Tajudin & Noraini Idris, 2013) that found that the use of graphing calculators 
either during teaching, during assessment, or both caused an increase in achievement. Ellington 
(2006) conducted a meta-analysis of research that had investigated the achievement of students 
using Non-CAS Graphing Calculators and summarized, . .When calculators were included in 
testing and instruction, the procedural conceptual and overall achievement skills of students 
improved" (p. 2). 

The large effect sizes in this descriptive study suggest that the use of graphing calculators 
in math class and/or for the assessment explains some of the variance in the average scale scores. 
However, it is always unclear from this type of research whether the “use of graphing calculator” 
in class and during assessment is a proxy for characteristics of the students, the students’ home 
life, the socioeconomic status (SES) of the students’ parents/guardians, the quality of the school 
that the students attend, or the teachers who teach them. 

Suggestions for Further Research 

The use of graphing calculators on large-scale assessments such as NAEP, PISA, 
TIMMS, and the SAT could be manipulated in a quasi-experimental design. The international, 
cross-cultural studies using quasi-experimental and experimental designs — cited in the review of 
literature for this study — found that the secondary school student use of graphing calculators 
increased both learning and assessment scores. 

This descriptive study found that the combination of using a graphing calculator in math 
class and using a graphing calculator while taking the NAEP assessment resulted in higher 
achievement (consistent with experimental/quasi-experimental research reviewed). 
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Future Research Question: 

“If we increase the number of 12 th -grade US students who use graphing calculators 
both in math class and during the NAEP assessment, will the NAEP average scale 
score increase?” 

END NOTE: The authors suggest that the two questions used in this study be continued in 
future NAEP twelfth-grade mathematics assessments. It would be helpful if these questions 
could be added to the NAEP 8 th -grade mathematics assessments. Further, the responses to these 
two questions would also be valuable for TIMMS and PISA for use by cross-cultural researchers 


and policy-makers. 
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Table 1. How Often Do You Use These Different Kinds of Calculators in Math 
Class? Graphing. [M8 17601] 


Never Use 

Sometimes Not Often 

Usually Use 

18% 

21% 

61% 

M 

SD 

M 

SD 

M 

SD 

137 

29 

145 

34 

165 

32 


Scale Range 0=300 


Table 2. Difference In Average Scale Scores Between Variables, For Use Graphing 
Calculator In Math Class [M8 17601] National Public, 2013 



Never use 
(137) 

Sometimes but 
not often 
(145) 

Usually use 
(165) 

Never use 
(137) 


< 

Diff = -8 
P-value = 0.0000 
Family size = 3 

< 

Diff =-28 
P-value = 
0.0000 

Family size = 3 

Sometimes but not 
often 
(145) 

> 

Diff = 8 
P-value 
0.0000 

Family size = 3 


< 

Diff =-20 
P-value = 
0.0000 

Family size = 3 

Usually use 
(165) 

> 

Diff =28 

P-value 

0.0000 

Family size = 3 

> 

Diff = 20 
P-value = 0.0000 
Family size = 3 


LEGEND: 

< 

Significantly lower. 

> 

Significantly higher. 

X 

No significant difference. 

NOTE: All comparisons are independent tests with an alpha level of 0.05 adjusted for 
multiple pairwise comparisons according to the False Discovery Rate procedure. For 
comparisons between two jurisdictions, a dependent test is performed for cases where 
one jurisdiction is contained in the other. For more detailed information about the 
procedures and family sizes please see the Help document. 


Table created by NAEP Data Explorer (NCES, 2014a) 
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Table 3. Effect Sizes of Differences in Scores by Calculator Use During Math Class 


Type of Calculator 

Cohen’s d Effect Size 

Usually Use 

Never Use 

d = 0.90 

Usually Use 

Sometimes Not Often 

d = 0.60 

Sometimes Not Often 

Never Use 

d = 0.26 


Table 4. Kind of Calculator Used for Test [M830201] by Average Scale Score 


None 


NAEP Provided 

Scientific Not 
Graphing 

Graphing 

35%* 


39%* 

8%* 

17%* 

M 

SD 

M 

SD 

M 

SD 

M 

SD 

153 

33 

147 

32 

146 

31 

171 

33 


Scale: 0-300; Percentages do not sum to 100% due to rounding. 
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Table 5. Mathematics, Grade 12. Difference in Average Scale Scores Between Variables, for 
Kind of Calculator Used for Test [M830201] National public, 2013 



None 

(153) 

NAEP- 

provided 

(147) 

Scientific not 
graph 
(146) 

Graphing 

(171) 

None 

(153) 


> 

Diff =6 
P-value = 
0.0000 

Family size = 6 

> 

Diff = 7 

P-value = 0.0000 
Family size = 6 

< 

Diff =-18 
P-value = 0.0000 
Family size = 6 

NAEP-provided 

(147) 

< 

Diff = -6 
P-value = 
0.0000 

Family size = 6 


X 

Diff = 1 

P-value = 0.4592 
Family size = 6 

< 

Diff =-24 
P-value = 0.0000 
Family size = 6 

Scientific not 
graph 
(146) 

< 

Diff = -7 
P-value = 
0.0000 

Family size = 6 

X 

Diff = -1 
P-value = 
0.4592 

Family size = 6 


< 

Diff =-25 
P-value = 0.0000 
Family size = 6 

Graphing 

(171) 

> 

Diff = 18 
P-value = 
0.0000 

Family size = 6 

> 

Diff =24 
P-value = 
0.0000 

Family size = 6 

> 

Diff =25 
P-value = 0.0000 
Family size = 6 


LEGEND: 

< 

Significantly lower. 

> 

Significantly higher. 

X 

No significant difference. 

NOTE: All comparisons are independent tests with an alpha level of 0.05 adjusted for multiple pairwise 
comparisons according to the False Discovery Rate procedure. For comparisons between two jurisdictions, a 
dependent test is performed for cases where one jurisdiction is contained in the other. For more detailed 
information about the procedures and family sizes please see the Help document. 


Table created by NAEP Data Explorer (NCES, 2014a) 




CALCULATOR USE TWELFTH- GRADE NAEP MATHEMATICS 


24 


Table 6. Effect Sizes of Differences in Scores by Calculator Use During NAEP Test 


Type of Calculator 

Cohen’s cl Effect Size 

Graphing Calculator 

No Calculator 

cl =0.55 

Graphing Calculator 

NAEP-Provided 

cl = 0.1 A 

Graphing Calculator 

Scientific Non- 
Graphing 

cl = 0. 78 


Table 7. Average Mathematics Scale Score by Kind of Calculator Used for NAEP Test 
[M830201] by Use Graphing Calculator in Math Class [M817601] 


Use Graphing 
Calculator in 
Math Class 

None 

NAEP-provided 

Scientific Not 
Graph 

Graphing 

Average 

scale 

score 

Standard 

deviation 

Average 

scale 

score 

Standard 

deviation 

Average 

scale 

score 

Standard 

deviation 

Average 

scale 

score 

Standard 

deviation 

Never use 

137 

29 

135 

29 

142 

27 

139 

38 

37% 

42% 

17% 

5% 

Sometimes 
but not often 

147 

34 

140 

33 

146 

33 

159 

37 

35% 

39% 

14% 

11% 

Usually use 

166 

31 

159 

32 

153 

33 

177 

29 

34% 

33% 

5% 

28% 


Table generated by NAEP Data Tool (NCES, 2014a). Format modified to show percentages. 


Row percentages may not sum to 100% due to rounding. 













































